M3I: A Framework for Mobile Multimodal Interaction
نویسندگان
چکیده
We present M3I, an extensive multimodal interaction framework for mobile devices, which simplifies and accelerates the creation of multimodal applications for prototyping and research. It provides an abstraction of information representations in different communication channels, unifies access to implicit and explicit information, and wires together the logic behind context-sensitive modality switches by a rule-based approach. In this paper, we present the structure and major features of our framework, and show exemplary implementations of interaction modalities with help of M3I.
منابع مشابه
M3I in a Pedestrian Navigation & Exploration System
In this paper, we describe a near-complete Pocket PC implementation of a Mobile Multi-Modal Interaction (M3I) platform for pedestrian navigation. The platform is designed to easily support indoor and outdoor navigation tasks, and uses the combination of several modalities for presentation output and user input. Whereas 2D/3D-graphics and synthesized speech are used to present useful information...
متن کاملMMIR Framework: Multimodal Mobile Interaction and Rendering
In this article, we present the MMIR (Multimodal Mobile Interaction and Rendering) framework that is targeted at building lightweight dialog systems. The framework is geared towards multimodal interaction development for applications where only limited resources are available, e.g. so that they can run self-contained even on mobile devices. The framework utilizes SCXML for modeling application-...
متن کاملSupporting Out-of-turn Interactions in a Multimodal Web Interface
Multimodal interfaces are becoming increasingly important with the advent of mobile devices, accessibility considerations, and novel software technologies that combine diverse interaction media. This article investigates systems support for web browsing in a multimodal interface. Specifically, we outline the design and implementation of a software framework that integrates hyperlink and speech ...
متن کاملSupporting Mobile Multimodal Interaction with a Rule-Based Framework
Multimodality can make (especially mobile) device interaction more efficient. Sensors and communication capabilities of modern smartphones and tablets lay the technical basis for its implementation. Still, mobile platforms do not make multimodal interaction support trivial. Building multimodal applications requires various APIs with different paradigms, high-level interpretation of contextual d...
متن کاملFlexible multimodal human-machine interaction in mobile environments
This article describes requirements and a prototype system for a flexible multimodal human-machine interaction in two substantially different mobile environments, namely pedestrian and car. The system allows an integrated trip planning using multimodal input and output. Motivated by the specific safety and privacy requirements in both environments, we present a framework for flexible modality c...
متن کامل